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ABSTRACT 






The employment of unmanned aerial vehicles (UAVs) in combat operations has 
demonstrated that UAVs can effectively provide surveillance, reconnaissance, and target 
acquisition support in place of manned aircraft. However, the Pioneer UAV, currently 
employed by the U.S. Navy and Marine Corps, has an unacceptable mishap rate. Half of 
the UAV mishaps are attributable in part to human factors causes. This points to a 
requirement for developing tailored intervention strategies. This study develops a 
stochastic simulation model of UAV mishaps to be used for the evaluation of human 
factor initiatives in terms of budgetary cost and mission readiness. It determines that 
electro-mechanically caused mishaps cost approximately the same as human factors 
mishaps. However, in comparison, human factors mishaps degrade mission readiness 
significantly. Intervention strategies need to address unsafe acts by the operator, unsafe 
conditions for flight operations, and unsafe supervision. The study recommends the 
following intervention measures: the use of system simulators; the implementation of 
improved aircrew coordination training; and the stabilization of personnel assignments. 
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EXECUTIVE SUMMARY 



The employment of unmanned aerial vehicles (UAVs) by the Israeli Defense 
Forces during the Peace for Galilee campaign in 1982, and by United States forces during 
Operations Desert Shield and Desert Storm in 1990-91, provides a proof of concept that 
relatively low cost UAVs can provide effective surveillance, reconnaissance, target 
acquisition and fire support adjustment missions. The United States has employed UAVs 
subsequently in operations in Somalia, Haiti, Bosnia, and most recently in Kosovo. The 
Department of the Navy’s doctrine of Operational Maneuver from the Sea will place 
greater reliance on aerial battlefield surveillance from "over the horizon" or expeditionary 
sites ashore in order to control a greater operational area with less force concentration. 
However, the current Naval UAV platform, the Pioneer, is beset by an unacceptable 
mishap rate. Since its fielding in operational units in 1986, the Pioneer Class A flight 
mishap rate is 385 mishaps per 100,000 flight hours. This stands in stark contrast to that 
of manned Naval Aviation where the rate is approximately two Class A flight mishaps 
per 100,000 flight hours. 

Schmidt & Parker (1995), while working at the Naval Safety Center in Norfolk, 
Virginia, have identified that human factors related issues cause half of the Naval UAV 
mishaps. Seagle (1997) applies the Human Factors Analysis and Classification System 
(HFACS) taxonomy to UAV mishap reports in order to improve human factors mishap 
investigation, reporting and analysis. The HFACS taxonomy, based upon the Reason 
(1990) "Swiss Cheese" model of accident causation, captures the latent conditions that 
"set the stage" for active failures that lead to mishaps. 

xvii 



This study refines the coding of UAV mishaps in accordance with the HFACS 
taxonomy. Once the mishaps are parsed, the factors influencing the mishap rate and their 
resultant costs are computed. These results are used to conduct a stochastic model 
simulation of annual UAV flight operations in order to isolate those categories of mishaps 
that contribute most to increased budgetary costs and decreased mission readiness. The 
study places UAV mishap reports from FY86 to FY98 into categories using the HFACS 
taxonomy. Particular emphasis is placed on the FY93 to FY98 period when Naval UAVs 
first came under the cognizance of the Naval Aviation Safety Program. Then, mishap 
rates and probabilities are estimated for the various categories. Thereupon, these 
probability distributions are used as the input to a stochastic model for the simulation of 
annual flight operations. The model output is the annual mishap costs associated with a 
particular category, and a resulting mission readiness index. 

Intervention strategies are assigned to associated category of mishap causation. 
Comparing the annual mishap cost, readiness index, and the feasibility associated with a 
particular intervention strategy, Fleet users and program managers can determine what 
intervention strategies are most appropriate. Some strategies are specific to the Pioneer 
system, for example, an engine remanufacturing or upgrading, or an electronic 
weatherproofing modification. Other strategies, such as improved aircrew coordination 
training and personnel assignment stabilization, transcend the Pioneer system, and are 
applicable to all follow-on UAV configurations. 

The mishap categorization phase of this study, data coding and classification, was 
conducted independently of previous studies (Schmidt & Parker, 1995; Seagle, 1997). 
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The relative frequency of mishap causes agrees with the previous classifications, and 
validates their findings. Even though mishap reports remain vague, and lack granularity 
at identifying lower, root causes, there is general agreement in the classification of the 
mishap in each independent analysis. 

The second phase of the study, stochastic model simulation of annual flight 
operations, finds that electro-mechanical causes have a low impact on mission readiness 
although they account for approximately one-quarter of UAV mishaps. On the other 
hand, unsafe acts, unsafe conditions for flight operations, and unsafe supervision have a 
more significant impact on equipment repair and replacement costs, and mission 
readiness. The study concludes that addressing the human factors related issues through 
increased aircrew coordination training, the use of simulators for mission rehearsal, 
personnel stabilization, and the development of a UAV career path, will have a greater 
impact on controlling cost and improving readiness. The study also generates the 
budgetary costs of selected mishap categories. These costs can be compared to the cost of 
implementing a particular intervention strategy. Thereupon, the program manager can 
chose the most appropriate strategies for implementation. While in isolation, no single 
intervention strategy will eliminate mishap occurrences; their implementation will 
increase the density of the "safety net" surrounding UAV operations. 
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I. INTRODUCTION 



A. OVERVIEW 

From the advent of modem military operations through the present day, successful 
commanders have relied on knowledge of the battlefield and intelligence of the enemy’s 
disposition in order to employ their force to achieve victory (Marine Aviation Weapons 
and Tactics Squadron One [MAWTS-1], 1997). During the 19 th century, in the American 
Civil War and the Franco-Prussian War, hot air balloons were employed in order to get a 
bird’s eye view of the battlefield. They added a third dimension to the intelligence 
collecting effort. Early in the 20 th century, the invention of the airplane replaced the 
restrictive and vulnerable use of balloons. During the First World War, manned aircraft 
were employed as airborne intelligence collectors. Later, in the 1960’s in response to 
increasingly lethal air defense networks and enormously developing technology, 
reconnaissance satellites have taken over the role of aerial surveillance at the strategic and 
operational levels of warfare. Presently, Unmanned Aerial Vehicles (UAVs) are used to 
augment satellite systems by providing near real-time, tactical aerial reconnaissance. 
However, the current Naval UAV, the Pioneer, is beset by an unacceptable mishap rate. 
This thesis addresses several potential intervention strategies designed to mitigate these 
mishaps, lower their budgetary cost and improve operational readiness. 

R. AERIAL RECONNAISSANCE 

The U-2 "spy plane" was developed during the early 1950s in order to monitor 
Soviet ICBM development and deployment (Jones, 1997). The U.S. conducted U-2 over 
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flights of the Soviet Union since 1955. On 2 May 1960, Francis Gary Powers was shot 
down in a U-2 over the Soviet Union by an SA-2 "Guideline” missile. The shoot down of 
Powers, and the failed attempt at covering up the mission, proved to be a devastating 
blow to the U.S.’s international prestige. A second U-2 was shot down over Cuba during 
the Missile Crisis on 27 October 1962, while attempting to determine the status of Soviet 
nuclear missiles. Consequently, the nation became increasingly wary of manned 
reconnaissance. However, the only reconnaissance systems in development were the SR- 
71 Blackbird and the CORONA spy-satellite. Both of these systems were designed for 
strategic level reconnaissance only. 

The U.S. made an initial serious attempt at an airborne tactical reconnaissance 
UAV during the Vietnam War (Jones, 1997). The U.S. Air Force experimented with 
launching the Teledyne-Ryan developed Lightning Bug UAV from MC- 130’s in order to 
conduct aerial reconnaissance. By the end of the war, the Lightning Bug UAV had grown 
into a vehicle for providing real-time video, electronic intelligence (ELINT) collection, 
electronic countermeasures (ECM), communications intelligence (COMINT) and 
psychological operations (PSYOPS) leaflet dropping. However interest in UAVs waned 
as the war wound down, and the Department of Defense (DoD) trimmed budgets and 
force structure. 

Nevertheless, U.S. lessons learned in Vietnam did not go unheeded. During that 
same period, Israel Aircraft Industries (IAI) began the successful development UAVs for 
the Israeli Defense Forces (IDF) (Kumar, 1997). These Israeli developed and 
manufactured UAVs were first employed in combat in 1982 during the Peace for Galilee 
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campaign in Lebanon. During an attack against Syrian forces in the Bekaa Valley, the 
Israeli Air Force (IAF) launched decoy missiles to stimulate the Syrian air defense 
system. As the Syrians responded to this perceived attack, Israeli Mastiff and Scout 
UAVs with electro-optical and radar-detecting payloads were able to locate and target 
Syrian missile sites. Once target locations were confirmed, and during the Syrian missile 
reload cycle, the IAF launched A-4, F-4 and KFIR aircraft to attack these targets. The 
Israelis destroyed 19 SAM batteries and 86 MiG aircraft while only losing one aircraft of 
their own, effectively dismembering the entire Syrian air defense network. 

The early 1980s saw a resurgence in enthusiasm for UAVs within DoD as a result 
of the Israeli’s successful UAV employment in Lebanon. U.S. peacekeeping operations in 
Beirut (1982-3), Operation Urgent Fury (Grenada, 1983), Operation Eldorado Canyon 
(Libya, 1986) all highlighted a requirement for an inexpensive, over the horizon, 
unmanned reconnaissance capability for the on-scene tactical commander. In July 1985, 
the Secretary of the Navy, John Lehman directed the expeditious acquisition of UAVs for 
fleet operations. By December 1985, the U.S. Navy procured the Pioneer UAV system 
developed by LAI. The Pioneer UAV is the next generation UAV following the Mastiff 
that was employed in the Peace for Galilee campaign. The U.S. Navy and Marine Corps 
began establishing UAV units in 1986. The first Pioneer overseas deployment occurred 
in December 1986, aboard the U.S.S. Iowa (Pioneer UAV Inc., 1999). 

The first time the Pioneer UAV saw combat action was in Operations Desert 
Shield and Desert Storm (Melson, Englander & Dawson, 1992). During that conflict, six 
UAV units were deployed to the Kuwaiti Theater of Operations (KTO) - two Navy, three 
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Marine Corps and one Army. Together, the units flew 336 missions, accruing 985 flight 
hours. Only three UAVs were hit by enemy anti-aircraft artillery (AAA), resulting in the 
loss of only one UAV from enemy action. General Walter Boomer, USMC (ret.) who 
commanded the Marine Expeditionary Force (MEF) in the Kuwaiti theater summed up 
the effect of UAV employment during the war when he said, "The Pioneer UAV was the 
most significant intelligence collection source within I MEF." 

Pioneer UAV units were later deployed for Operations Restore and Continue 
Hope in Somalia, Operation Uphold Democracy in Haiti, and Operation Joint Endeavor 
in Bosnia. Their successful mission performance reinforced the potential of exploiting 
this technology to support future combat operations and operations other than war 
(OOTW) (Jenkins, 1998). The Department of the Navy’s doctrine of "Operational 
Maneuver from the Sea" (OMFTS) will place greater reliance on battlefield surveillance 
from "over the horizon” and expeditionary sites ashore in order to control a greater 
operational area with less force concentration. Relatively inexpensive reconnaissance 
UAVs with the ability to conduct missions in hostile airspace, without putting pilots into 
harm’s way is a key element of OMFTS (Marine Corps Combat Development Command 
[MCCDC], 1999). 

C. THE PIONEER UAV SYSTEM 

A Pioneer UAV system consists of five Pioneer air vehicles, a ground control 
station (GCS), a portable control station (PCS), a tracking communications unit (TCU), a 
data link, two remote receiver stations (RRS) and a reconnaissance payload. The system 
can be operated aboard specially configured U.S.S. Austin Class Landing Platform Dock 
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(LPD-4) ships or from prepared airstrips ashore. While UAVs do not carry any ordnance, 
they can perform as forward observers of indirect fire support assets - offensive attack 
aviation, artillery and naval surface fire support. Figure 1 is a representation of how 
UAVs can be deployed during an amphibious operation. 




LRE Launch Recovery Element 
MCE Mission Control Element 
LOS Line of Sight 






Figure 1: Pioneer UAV Concept of Operations 
Within DoD, only the U.S. Navy, Marine Corps and the Defense UAV Training 
Command (DUTC) operate the Pioneer UAV (MAWTS-1, 1997). The Navy's UAV unit 



is VC-6, headquartered in Norfolk, Virginia; however, the UAV elements are located at 



Webster Field, Patuxet River, Maryland. From there, they deploy as detachments with a 
complete Pioneer system aboard ship. There are two Marine Corps UAV units. VMU-1 
is located in Twenty-Nine Palms, California, and VMU-2 is headquartered at MCAS 
Cherry Point, North Carolina. The training command, DUTC, is located in Fort 
Huachuca, Arizona. The U.S. Army deactivated their Pioneer UAV units in anticipation 
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of procuring the Hunter (IAI) or Outrider (Alliant Techsystems) UAV system. However, 
both programs were cancelled because they failed to meet desired size, weight and 
corrosion control specifications (Sherman, 1998). 

Pioneer UAV units are currently tasked to provide the following missions: 

a) Reconnaissance, surveillance and target acquisition (RSTA). 

b) Adjusting indirect fire (Artillery, Naval Surface Fire Support). 

c) Collect Battle Damage Assessment (BDA). 

d) Support security operations (e.g., convoy escort, monitor enemy 

avenues of approach and named areas of interest (NAIs). 

The term "unmanned" is a misnomer when applied to the Unmanned Aerial 
Vehicle system because UAV operations involve many remote participants. The essential 
members of a UAV crew include a Mission Commander, an Internal Pilot, a Payload 
Operator and an External Pilot. Additionally launch and recovery teams and maintenance 
personnel will be involved in flight operations (Joint UAV Training Operations 
Procedures & Standardization [JUAVTOPS], 1997). A detailed description of the 
Pioneer UAV system, capabilities and crew responsibilities is attached as Appendix A. 

D. PIONEER UAV SAFETY RECORD 

Schmidt & Parker (1995), of the Naval Safety Center (NSC), begin the initial 
effort at improving UAV operational readiness by focusing on mishap prevention. Their 
study examines the 107 mishaps that occurred between 1986 and 1993. The breakdown 
of causal factors is illustrated in Figure 2. This research indicates a significant number 
(approximately 59%) of mishaps occur as a result of electromechanical problems. 
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Human error accounts for nearly one-third of these mishaps. Their research indicates the 
following factors present significant safety concerns: crew selection and training; 
aeromedical readiness; pilot proficiency/currency; personnel shortages; operational tempo 
(OPTEMPO); human error in teamwork and aircraft control. 




Launch Error 
10% 



Misc. 



Mechanical Failure 
10 % 



Engine Failure 
25% 



Landing Error 
22% 



Electrical Failure 
24% 



Figure 2: FY 86 - FY 94 Naval Pioneer UAV Mishap Causal Factors (Schmidt & Parker, 1995) 
Schmidt & Parker (1995) recommend the following corrective actions be taken to 
decrease the mishap rate: aeromedical screening and monitoring; criteria based selection 
process; UAV crew coordination training; take off, landing and external pilot (EP) 
simulators; inclusion into OPNAVINST 3710.7P NATOPS oversight and improved 
human systems integration. These recommendations have been implemented with 
varying degrees of success. Aeromedical screening is conducted prior to being assigned 
to the UAV community. In addition, aptitude testing is conducted to assess the flight 
potential of incoming aircrew. Aircrew coordination training (ACT) is currently in 
progress and constantly evolving. A take off and landing drone was developed to enable 
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EPs to fly an air vehicle without having the entire system activated. In FY 94, UAV 
flight operations were included in the Naval Aviation Safety Program (OPNAVINST 
3750.6Q). 

These improvements have marginally decreased the mishap rate. However, 
improvement is still required. Initially, aeromedical screening takes place but a flight 
surgeon is not assigned to UAV units to continue aeromedical education and crew 
monitoring. The lack of operational experience and personnel stability in UAV units 
limits ACT because lessons learned are difficult to capture and implement. Additionally, 
emergency action simulation drills are not possible because there is no crew simulator. 
Air vehicle drones, called MiGs by their crews, do not have the same aerodynamic 
characteristics of the Pioneer air vehicle. Thus, while the MiG training is effective in 
training the EPs in basic flight controls and procedures, the discrepancies between its 
response and that of the real Pioneer vehicle may reinforce improper handling and result 
in a negative learning experience. 

Seagle (1997) continues the work of Schmidt & Parker. He applies the Human 
Factors Mishap Classification System (HFACS) to analyze UAV mishaps from 1986 to 
1997. His research studies the 203 mishap investigation reports from 1986 through 1997, 
determining that 88 include human related causal factors. His categorization of mishaps 
is illustrated in Table 1. His work provides insight into the cause of human factors 
related mishaps and clarifies the broad, generalized "human error" labels. Seagle goes 
further by demonstrating that although the primary cause of an accident may have been 
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electro-mechanical in nature, a latent cause was due potentially to human factors, either 
contributing to the mishap, or failing to correct a condition that led to the accident. 



CAUSAL FACTOR 




CODE 


# 


FREQ 


Unsafe Act 




UA 


52 


59.1% 




Intended 


UAI 


6 


6.8% 




Mistake 


UAIM 


34 


38.6% 




Violation 


UAIV 


6 


6.8% 




Unintended 


UAU 


46 


52.3% 




Slip 


UAUS 


2 


2.2% 




Lapse 


UAUL 


14 


15.9% 


Unsafe Condition 




UC 


40 


45.5% 




Aeromedical 


UCA 


18 


20.4% 




CRM 


UCC 


24 


27.2% 




Readiness 


UCV 


6 


6.8% 


Unsafe Supervision 




US 


54 


61.4 




Unforeseen 


USU 


30 


34.1% 




Foreseen 


USF 


41 


46.5% 


Human Factors 




HF 


88 





Table 1 : FY86 - FY97 UAV Mishaps Parsed by Causal Category (SeagLe, 1997) 

Seagle confirms Schmidt & Parker’s recommendations for a full crew simulator to 
enhance the rehearsal of ACT and flight emergency drills. He also reports a trend in the 
following categories: loss of situational awareness, lack or loss of depth perception, 
visual illusions, self-medication and fatigue. In 1996, the Navy’s Bureau of Medicine 
(BUMED) incorporated medical standards for UAV aircrew to address these factors. 
Seagle also proposes an automatic, "hands off' landing system for runway arrestment and 
embarked net recoveries with an override capability for the EP or IP in the case of 
degraded operations. He attributes the lack of urgency in implementation of these efforts 
to the UAV community having no "champion" at the flag officer level. The relatively 
inexperienced and young officers who serve in the UAV community have not reached (or 
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may never reach) the rank where they can affect change, unlike officers in manned 
aviation careers. 

E. RESEARCH OBJECTIVE 

The objective of this study is to further identify the causal factors resulting in the 
unacceptable number of UAV mishaps. Particular emphasis is placed on human factors 
related mishaps. Once identified the study constructs a stochastic model to evaluate 
mishap intervention initiatives with the goal of mishap reduction in terms of cost and 
mission readiness. The results of the model are presented to allow decision-makers to 
focus on specific accident causation categories and to choose the most efficient and 
effective intervention strategies for further development. 

F. STATEMENT OF THE PROBLEM 

Since its fielding in 1986, the Pioneer has accumulated a Class A mishap rate of 
385 mishaps per 100,000 flight hours. When this is compared to the manned aviation 
Class A mishap rate of approximately two mishaps per 100,000 flight hours, one sees the 
Homeric proportions of this unacceptable situation. The excessive UAV mishap rate 
translates into significant budgetary cost, degradation in mission readiness, and a 
perception of unreliability by fleet users and those whom they support. Steps must be 
taken to bridge the gap between conceptual capability and actual performance. The 
bottom line is to achieve a dramatic reduction in the UAV mishap rate to an acceptable 
level, to sustain mission readiness, and to minimize mishap costs. Specific research goals 
include the following: 
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1. The classification of UAV mishaps under the current aviation mishap 
taxonomy. Additionally, the identification the human factors characteristics that 
significantly impact the UAV mishap rate. 

2. The development of a stochastic model of UAV mishaps which can be used to 
accurately represent mishap occurrences. 

3. The use of the model to analyze the effects of mishap reduction intervention 
strategies. 

4. The identification of the impact of a particular intervention strategy on cost 
savings and mission readiness improvement. 

G. DEFINITIONS 

Mishap . A naval mishap is an unplanned event or series of events directly 
involving naval aircraft, which results in $10,000 or greater cumulative damage to naval 
aircraft or personnel injury. Aviation flight mishaps are divided into one of three 
categories based on the severity of the damage, and the cost incurred by the mishap. The 
definitions governing each class of mishap is given in Naval Publication OPNAVINST 
3750.6Q. 

Class A Mishap : A class A mishap occurs when the total amount of damage 
exceeds $1,000,000 or if the air vehicle is destroyed. Because the total cost of a Pioneer 
UAV is approximately $1.1 million, a class A mishap will only occur if the UAV is 
damaged beyond repair or lost. Class A mishaps include being lost at sea, or destruction 
of the air vehicle if it forcefully impacts terrain. Loss of a UAV during a combat mission 
is not classified as a class A mishap, but rather, a combat loss. 
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Class B Mishap : This category is used when the total cost of damage is at least 
$200,000 but less than $1,000,000. Usually, a UAV class B mishap occurs when there is 
serious damage to the air vehicle, or if the payload is damaged. The cost of the 
surveillance camera alone ranges from $300,000 to $800,000 depending on whether it is 
the day EO or night FLIR camera. The variance of the cost depends upon whether the 
payload is repairable or permanently damaged. 

Class C Mishap : This category is used when the total cost of damage is at least 
$10,000 but less than $200,000. For UAV mishaps, this situation occurs when there is a 
small amount of damage to the air vehicle possibly from a hard landing, or striking 
another object. Usually, swapping out or remanufacturing parts can repair these types of 
mishaps, keeping costs low. 

H. SCOPE AND LIMITATIONS 

The scope of this research is limited to the Defense UAV Training Command, and 
U.S. Navy and Marine Corps fleet Pioneer UAV squadrons. However, the results of this 
analysis will be pertinent to all UAV operations of current and future systems that 
integrate a human component to conduct the mission. Detailed mishap analysis is 
conducted from FY93 through FY98. In FY93, UAVs were incorporated into the Naval 
Aviation Safety system, and mishap reporting procedures were standardized. Prior to 
FY93, UAV mishap reports contained limited descriptive information, rendering detailed 
analysis nearly impossible. Mishap causes were typically described as pilot error, 
electrical or mechanical failure with little amplifying information. 
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The contents of the thesis are presented in the following order: Chapter Two is a 
literature review of three human error causation models. They serve to provide 
background to the HFACS model used by the NSC, which in turn is discussed in detail 
with specific examples of UAV applications. Chapter Two also discusses accident 
investigations and analysis, mishap intervention strategies, and stochastic modeling. 
Chapter Three describes the methods used to estimate statistical parameters, which are 
inputs to the stochastic model. Further, the simulation methodology is developed. 
Chapter Four includes the mishap database construction, parameter estimates and the 
output of the stochastic model simulation. Conclusions and recommended courses of 
action are presented in Chapter Five. 
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II. LITERATURE REVIEW 



A. ACCIDENT CAUSATION THEORIES 

The Asia-Pacific Safety Magazine (March, 1995), published by the Australian 
Bureau of Air Safety Investigations, reports "between 70% and 80% of [mishap] 
occurrences contain a human factors element." The article continues, "Indeed it can be 
argued that human factors are involved in all occurrences." In a system designed, built, 
maintained and operated by humans, only a fraction of one percent of the mishaps can be 
attributed to factors beyond human control (Bruggink, 1996). 

Because of society's obvious goal to reduce the number of aviation mishaps and 
eventually their overall prevention, much research has been dedicated to determining 
accident causation, and developing a safe environment where the probability of having an 
accident is minimized (Harle, 1993). Nevertheless, the results of aviation accident 
investigations often limit conclusions to phrases such as "pilot error," "failure to see and 
avoid," "improper use of controls," or "failure to observe and adhere to established 
standard operating procedures (SOPs). 

"However, effective accident intervention and prevention requires more than 
identifying who is culpable. In order to implement a safety conscious program, a better 
understanding of the context in which these individuals faced accident conducive 
circumstances is required. 

Within the last thirty years, three accident causation theories stand out for their 
merits in dissecting the myriad of contributing causal factors that create the context of an 
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incident. These three theories are Edwards (1977) "SHEL" model; Helmreichs (1990) 
model; and the Reasons (1990) "Swiss Cheese" model. Each of the three theories 
attempts to explore the latent or removed factors that influence the mishap. The main 
focus is on the "chain of events" and surrounding circumstances that lead to an accident. 

1. The SHEL Model 

The "SHEL" (Software, Hardware, Environment and Liveware) model is first 
introduced by Edwards (1972) and later modified by Hawkins (1984). The SHEL model 
places emphasis on the human being and its interfaces with the other components of the 
man-machine-environment system. Each component of the SHEL model represents the 
components of a modem technological system as depicted in Figure 3. 




Figure 3: The SHEL Model 

The human operator is set at the center of the model, and must interact with each 
of the four external components. The edges of the blocks are not simple or straight, 
indicating that the human must be matched with each component in order to function 
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properly. A mismatch between blocks, or an improper fit is a potential cause for human 
error. In order to analyze the human factors aspects of an environment, one must look at 
both the individual component blocks, and also their interface (Harle, 1993). The 
following is a brief description of each component of the model: 

a) Human Operator 

The human operator is the hub of the SHEL model. As such, analysis of 
the individual must incorporate four categories: physical, physiological, psychological, 
and psychosocial. Physically, one must determine if the individual is capable of 
performing the required task, and if there are any impediments or limitations to successful 
task performance. Physiologically, an individual must be prepared to conduct the task. 
This category includes the items of proper nutrition, alcohol or drug use, tobacco use, 
stress and fatigue, and the effect these have on an individual or crew’s ability to perform 
and make appropriate decisions. Psychologically, an individual must be capable of 
mentally executing the task. Knowledge of what is required, and the confidence to 
perform the task must be established. Moreover, the workload must be appropriate to an 
individual’s information processing and attention capabilities. Finally, psychosocial 
factors impact human performance. Stress, pressure from a supervisor, the workplace 
climate and personal issues all influence ones reaction in a potentially dangerous 
situation. 



b) Liveware 

The liveware interface is the operator s relationship to the other individuals 



in the workplace. In aviation, this is often referred to as crew resource management 
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(CRM) and is addressed later. Liveware also relates to teamwork, morale and the overall 



command climate. Obviously, healthy interpersonal relationships among a crew, or 
between individuals and their supervisors are essential for a safe and effective work 
environment. 



c) Hardware 

The hardware interface addresses the human machine interface (HMI). 
The HMI includes workspace configuration, displays, controls, seat design and 
configuration, visibility and climatic conditions. The physical work environment impacts 
crew orientation, information processing, cognition and execution of the task. Similarities 
of component design and their physical layout can affect effective scan patterns, and 
facilitate correlation of input data. The hardware interface is the focus of ergonomic and 
anthropometrical study. 

d) Software 

The software interface is the relationship between the individual and all 
supporting systems found in the workplace. This category includes not only computer 
software design, but also regulations, manuals, checklists, and SOPs. These items must 
be user friendly and understandable to the human operator. Automation is also a 
significant contributor to the software human interface. An entirely automated system 
may have the tendency to breed complacency and boredom for an operator resulting in 
decreased vigilance. On the other hand, the lack of automation can cause task saturation 
for an operator in extreme situations, also resulting in an unsafe environment. 
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e) Environment 

The environmental interface is the relationship between the operator and 
the internal and external environment. The internal environment includes temperature, 
lighting, noise, vibration and air quality. The external environment includes visibility, 
weather and terrain. For military applications, the external environment includes the 
combat situation during which operations must be executed. The surrounding 
environment has significant impact on individual and crew motivation, attention, 
judgment and performance. 

2. The Helmreich Model 

Helmreich advised the International Civil Aviation Organization (ICAO) 
Commission of Inquiry on the human factors aspects of an Air Ontario flight accident that 
took place in Dryden, Ontario in 1990 (Zotov, 1996). The advantage of his theory is that 
it eliminates an opportunity for regulators and organizations to argue that their actions 
should not be discussed in the accident report. It is frequently argued that an accident is 
the end result of a chain of events: if a causal factor can be removed and the accident 
could still have occurred, then ipso facto that factor can not be causal. The Helmreich 
model illustrates that an accident is the accumulation of factors, rather than their chaining 
together, which affects a crew’s performance. 

The Helmreich model can be envisioned as a series of concentric circles 
surrounding an operator or crew. Each ring potentially influences the crew and may 
cause a degradation of performance. The four levels of influence are the regulatory 
environment, the organizational environment, the physical environment, and the crew 
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environment (see figure 4). Accidents occur as the result actions taken within the context 



of this crew environment. 




a) Regulatory Environment 

The regulatory environment is the guidance for conducting operations. 
Within Naval aviation, this includes NATOPS procedures, the Training and Readiness 
(T&R) manual, unit SOPs, operations orders and commanders guidance. These 
regulations guide supervisors on how to conduct mission planning and execution within 
certain limitations. Examples of these regulations are proficiency and currency 
requirements, crew rest and approved flight profiles. 

b) Organizational Environment 

The organizational environment includes crew composition and its 

performance. Consistent training, personnel stability, OPTEMPO and leadership 

contribute to the organizational environment. On the micro level, the interface between 
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operations and maintenance personnel, and on the macro level between a unit and is 
higher and adjacent units within the command and control structure greatly influence the 
organizational environment and the niche into which a crew fits. 

c ) Physical En vironment 

The physical environment is consistent with the internal and external 
environments of the SHEL model. It also includes the physical condition of equipment at 
the time of an incident, and what affect that had on mission performance. The model 
recognizes that the surroundings of an individual or crew have a significant impact on the 
vigilance and responsiveness of the operator. 

d) The Crew Environment 

The crew environment comprises the interpersonal coordination and 
communication within a crew. The model also extends the crew definition to include its 
interfaces with external control, such as air traffic (ATC) and enroute controllers. 
Anyone associated with the conduct of a mission is de facto part of the extended crew. 
The crew environment is also influenced by CRM. Finally, the crew is further broken 
down into individual components. Individuals possess their own performance strengths, 
weaknesses and vulnerabilities. 

The Helmreich model of accident causation is crew centered. However, instead of 
"blaming" the crew or individuals on the crew, it looks at the context of the accident that 
arises from the faulty actions or decisions by external agencies. These contributing 
factors, although potentially missing from the "chain of events" are equally influential in 
an accident occurrence (Zotov, 1996). 
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3. The "Swiss Cheese” Model 

The "Swiss Cheese" model is the outgrowth of research examining human error 
by Reason (1990). The model discusses the layers of defense used to protect against 
accident occurrences. Reason’s model identifies four layers that potentially contribute to 



an accident: organizational influences, unsafe supervision, unsafe conditions and 



individuals performing unsafe acts (see Figure 5). 




Figure 5: Reason’s Swiss Cheese Model 

In an ideal world, the defensive layers would be intact, preventing the "accident 
trajectory," as depicted in the diagram by the arrow, from passing through to the accident 
event. However, each layer has weaknesses and gaps that are revealed by the holes. In 
the real world, these holes are not fixed and static, otherwise they could be identified and 
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repaired. Reason (1998) describes the holes as dynamic and in a constant state of flux. 
Local conditions drive which defensive layer comes in and out of the frame at a particular 
time. 

Reason (1998) attributes the holes as either active failures, or latent conditions. 
Active failures are either violations or errors that occur in the immediate vicinity of the 
accident occurrence. They are performed by the operators - pilots, air traffic controllers, 
police officers, control room operators, maintenance personnel, and so on. The discovery 
and mitigation of this active failure immediately prior to the accident would most likely 
prevent the accident from happening. Twenty years ago, the discovery of this unsafe act 
would have ended an accident investigation. However in today's climate, unsafe acts are 
seen more as consequences than principle causes. It is recognized that people working in 
complex systems make errors or violate procedures for reasons that go beyond an 
individual. These causes are called latent conditions. 

Latent conditions can be present for many years before they combine with local 
circumstances and active failures to penetrate they systems layers of defense. Examples 
of latent conditions include poor design, improper training and supervision, undetected 
manufacturing defects or poor design, unworkable procedures, or improper automation. 
At the macro level, government, regulator or corporate policy shapes the organizational 
culture, creating the error producing factors within an individual environment. 

Latent conditions are present in all systems and are an inevitable part of the 
organizational culture. Latent conditions are not bad policy decisions, but can result from 
the demands of a limited budget or manpower management constraints. These latent 



23 



conditions can lie dormant for years and have no impact until they become manifested at 
a time where particular weaknesses in the defense become exposed. In contrast to active 
failures, which tend to be unique to a specific event, latent conditions can go 
unrecognized and can contribute to a number of different accidents. Latent conditions 
increase the likelihood of active failures through the creation of local factors allowing 
error and violations to occur (Reason, 1998). 

B. HUMAN FACTORS ANALYSIS AND CLASSIFICATION SYSTEM 

The Human Factors Analysis and Classification System (HFACS) taxonomy as 
developed by Shappell & Wiegmann (1997) at the Naval Safety Center (NSC), is based 
on Reason’s concept of human error in accident causation. HFACS also incorporates, 
albeit to a lesser extent, components of the SHEL and Helmreich model to define the 
context surrounding an accident. The HFACS taxonomy will be added to the next 
publication of OPNAVINST 3750 and become the standardized methodology adopted by 
the Naval Aviation Safety Program for human factors mishap investigation. HFACS 
attempts to capture the context in which an accident occurs by categorized failures into 
four separate tiers depicted in Figure 6. These tiers are organizational influences, unsafe 
supervision, unsafe conditions, and unsafe acts. For the purposes of this discussion, each 
tier is viewed through the context of naval UAV operations. 
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Figure 6: HFACS Mishap Causation Categories 



1. Organizational Influences 

The first HFACS tier of mishap causation is organizational influences. 
Organizational influences are often difficult to quantify and can rarely be tied as a 
specific cause to an accident. However, their existence certainly contributes to the 
circumstances surrounding an accident. The author discussed these influences with UAV 
crewmembers at VMU-1, VMU-2 and VC-6. Among aviators, there is a perception that a 
UAV tour shows a lack of competitiveness with one’s peers who are in a flying billet in 
their primary airframe. Being assigned to the UAV unit relegates them to the status of a 
second-class citizen. In contrast, several see the UAV field as an opportunity to excel in a 
challenging and developing community. As such, they channel their drive and energy to 
guarantee success. 

Another organizational influence is the lack of a UAV specialty and career path. 
Until 1996, UAV enlisted personnel did not have a specific rating or military operational 
specialty (MOS). As a result, personnel were assigned to a UAV unit with no previous 
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background, underwent training and left the unit three years later, never to return again. 
Initial accessions to UAV units were recruited by advertising for individuals whose hobby 
was remote control airplanes. Initially, a unit could be made up of truck drivers, 
mechanics, yeomen, or any combination of backgrounds. The establishment of enlisted 
specialties has improved the situation. Nevertheless, there are no officer specialties or 
career paths. It is still the case that an officer will only serve one tour in a UAV unit 
before leaving the community. Typically, all officers in a UAV unit, including the 
commanding officer, will be new to the community upon assignment (MAWTS-1, 1997). 

A third organizational influence within the UAV community is the perception by 
others in the aviation field. Besides the normal rivalries that exist among communities, 
aviators typically relegate UAV crews to being the lowest on the totem pole. Often, their 
inclusion in aviation planning and operations in regarded as a nuisance by those who do 
not understand the unique integration requirements that UAV operations must address. 
These organizational conditions, unhealthy at times, can create an environment 
susceptible to unsafe operations and potentially lead to mishap causation. 

2. Unsafe Supervision 

Unsafe supervision is the second HFACS tier that contributes to an actual mishap 
occurrence. Unsafe supervision can be both unforeseen and foreseen. Unforeseen unsafe 
supervision includes unrecognized unsafe operations, inadequate documentation and 
inadequate design. Unrecognized unsafe operations result from a supervisor not 
recognizing an unsafe act or condition. For example, a supervisor may not be aware of 
accumulated fatigue among aircrew or maintenance personnel in a unit. Instead of taking 
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corrective action, the situation is ignored. Also, by not being aware of influences on 
individuals, crew assignments and personnel management suffer. An example is the 
supervisor who is not aware that a crewmember has a sick spouse or child, a recent death 
in the family, or marital difficulty. As a result, the supervisor does not know to take these 
adverse mental conditions into account when making crew assignments. 

Inadequate documentation refers to unknown "bugs" in the system. Designed for 
combat, the Pioneer was not fully tested until Operation Desert Storm. Operational 
testing and training are designed to closely resemble combat, but cannot replicate it. 
During Operation Desert Storm , the Marine Pioneer units operated as part of a full scale 
Marine Expeditionary Force (MEF). A microwave antenna/transmitter was located at the 
UAV airstrip for communications. The microwave signal caused electro-mechanical 
interference (EMI) with the UAV uplink and downlink causing loss of vehicle control 
resulting in crashes on several occasions. Because the Pioneer had not previously 
operated in close proximity to this specific communications equipment, this condition 
was neither realized nor documented. 

Inadequate design is an extension of inadequate- documentation in that an 
inadequate condition exists that is unintentional. The engineers who designed the 
equipment may not have anticipated requirements for certain performance characteristics 
or capabilities. Although it may seem inconceivable for a naval UAV, the Pioneer was 
not built to fly through rain or visible moisture. It was designed by the Israeli Aircraft 
Industries (IAI) for operations in arid areas around Israel. It has a laminated wooden 
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propeller that delaminates in moisture, and the electronic components on the air vehicle 
are not waterproof. This inadequate design has been the cause of several UAV mishaps. 

Foreseen unsafe supervision is the mismanagement of individuals at the personal 
level. It includes the lack of or inadequate supervision, the failure to correct a known 
problem, or a supervisory violation. Foreseen unsafe supervision is a lack of leadership 
and guidance in a crew or in the entire unit that creates an unsafe situation for flight 
operations. Within a unit, this can be an underlying condition that is reflected in lack of 
discipline, operational focus or morale. Unsafe supervision is rarely an isolated instance, 
but is symptomatic of underlying conditions, which can all lead to a mishap. 

3. Unsafe Conditions 

Reason (1990) addresses the category of unsafe conditions in accident causation, 
however Shappell & Wiegmann’s (1997) HFACS taxonomy further subdivides these 
conditions into aeromedical, crew resource management and readiness violations. The 
following paragraphs characterize each of these subdivisions. Additionally, each 
subdivision is discussed within the context of UAV community issues. 

a) Aeromedical Conditions 

Aeromedical conditions include the physiological and mental condition of 
individuals, and their physical and mental limitations. The physiological condition of an 
individual includes the functioning of their sensory system and physical condition. A 
UAV crewmember may experience spatial disorientation or visual illusions caused by the 
remote controls of the air vehicle. Unlike manned aviation where pilots incorporate 
vestibular inputs with visual cues to perceive the attitude and motion of the aircraft, a 
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UAV operator does not have those inputs. In order to operate effectively as a 
crewmember, there are required sensory thresholds that must be maintained. In addition 
to the senses, other physical conditions can affect the crewmember. Though not 
exhaustive, medications, fatigue, change in circadian rhythm, hypoglycemia, use of 
alcohol, or vitamin deficiencies can cause a person to be physically unqualified to 
conduct a mission (Edwards, 1990). 

The adverse mental condition of a crewmember is also an aeromedical 
condition that can adversely affect mission performance. The effects of stress and mental 
workload on human performance are widely documented. Both can cause a loss of 
situational awareness, mental fatigue and task saturation. Additionally, personality traits 
and attitudes such as overconfidence, complacency, misplaced motivation, or a desire to 
please, cause an adverse mental state. Increased OPTEMPO, and associated family 
separations, financial concerns, operational or combat fatigue and competition among 
members of a crew, all potentially combine to create the adverse mental conditions that 
can cause an accident (Hawkins, 1987). The final aeromedical contribution to unsafe 
conditions is the physical and mental limitations of the crewmember. Individuals must be 
physically and mentally screened for selection as UAV aircrew in order to prevent the 
environment for the previously discussed conditions to occur. 

b) Crew Resource Management 

Jensen (1995) defines crew resource management (CRM) as the effective 
use of all resources (hardware, software, and liveware) to achieve safe and efficient flight 
operations. The resources that a crew manages are people (other crewmembers), 
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equipment (instruments and controls) and other items such as charts, checklists and 
operational manuals. UAV operations, as stated earlier, involve a crew of at least four, 
and normally more crewmembers. The competency and experience of these individuals, 
their supervision, their interpersonal communication and the remote aspect of the air 
vehicle from the operational control site combine to make unique CRM demands on a 
UAV crew. The crew must communicate via an intercom system (ICS) and 
crewmembers often cannot establish eye contact with one another. Additionally, not all 
crewmembers have access to flight instrument information. As the number of 
crewmembers goes up, the potential for conflicting interpretations of information also 
increases, potentially causing confusion and indecisive actions. 

c) Readiness 

Readiness violations refer to violations of standard operating procedures 
(SOPs), rules, and instructions designed to provide a safe operating environment for flight 
operations. Among other things, NATOPS defines regulations on crew rest, self- 
medication and alcohol consumption. In the Marine Corps, the Training and Readiness 
(T&R) Manuals (MCO 3500.21: Volumes I and VI), define crew proficiency and 
currency requirements for an individual conducting a specified mission. Violations of 
any of these regulations can create unsafe conditions for a mission. 

4. Unsafe Acts 

The fourth HFACS tier is unsafe acts, which can be classified as either intended 
or unintended. Intended acts are either mistakes or violations, whereas, unintended acts 
are either slips or lapses. The following paragraphs characterize each of these 
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subdivisions. Additionally, each subdivision is discussed within the context of UAV 
community issues. 

a) Intended Acts 

Mistakes are failures to formulate the correct intentions, and can result 
from shortcomings of perception, memory and cognition. In these cases, the intended 
action is wrong. Knowledge based mistakes are caused by failures to understand the 
situation and arriving at an incorrect course of action. In this situation a UAV operator 
may be saturated with raw information, and may not be able to process the data to 
formulate the correct actions. Also, because of inexperience, an operator does not have 
the capacity to assimilate the information given to formulate a correct response. 

In contrast, someone with more experience who misapplies a rule under 
certain conditions usually makes a rule-based error. A rule-based decision can be likened 
to if.. .then logic. These mistakes normally occur in one of three ways. The first is that a 
rule is followed, but exceptions to the situation are not noted. An example is a UAV 
operator who has been operating from a shore location and is now operating at sea. 
Emergency procedures that are learned and reinforced through training ashore may not be 
applicable or desirable at sea. A mistake occurs when the operator applies a land-based 
procedure at sea and an accident occurs. The second type of rule based mistake is when 
the if... part of the situation is misinterpreted, and the ...then action is inappropriately 
applied. And finally, the third type of rule based mistake is when the if. . . observation is 
correct and the ...then action is incorrectly applied. (Wickens, 1992) 
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In contrast to mistakes, violations are the willful breaking of rules or 
procedures. Violations can be categorized into routine or exceptional violations. A 
routine violation tends to be habitual in nature and is typical of an individual’s behavioral 
repertoire. An example of a UAV related routine violation is when a crewmember 
habitually fails to follow the unit SOP or NATOPS procedures. This can be as harmless 
as failing to brief a certain emergency procedure during a pre-flight brief. Lack of 
supervision and correction action acerbates the violation by passively reinforcing 
incorrect procedures. On the other hand, an exceptional violation is an isolated departure 
from authority, neither typical of the individual nor condoned by supervisors. The 
deliberate decision to ignore directions from an air controller is an example of an 
exceptional violation. 

b) Unintended Actions 

A slip is an unintended error in which the correct intention is incorrectly 
carried out, as opposed to a mistake where the incorrect intention is correctly carried out. 
For example, the internal pilot (IP) uses dials to control the speed and altitude of the air 
vehicle. Intending to adjust the UAV airspeed and inadvertently changing the altitude 
dial is an example of a slip. According to Wickens (1992), slips occur for three reasons: 
(1) the intended action involves a slight departure from the routine, frequently performed 
action; (2) some characteristics of the stimulus environment or action sequence closely 
relate to the inappropriate, but more frequent action; and (3) the action sequence is 
automated and therefore, not monitored closely by attention. 



32 



Finally, a lapse is the failure to carry out an action. Memory failure, 
memory overload, or interruption can cause a lapse. Prior to every UAV mission, all 
members of the UAV crew perform various tasks as outlined in a preflight checklist. If 
that sequence of events is interrupted unexpectantly, requiring the crew or an individual 
to divert attention and then come back to the pre-flight sequence, a particular step may be 
skipped. This momentary distraction can cause a lapse to occur with unknown 
consequences. 

C. ACCIDENT INVESTIGATIONS, ANALYSIS, AND REPORTS 

Bruggink’s (1996) research in civil aviation points out that accident reporting is 
too preoccupied with reactive, formal responses to stated accident causes. Typical 
investigations focus on the black and white elements of accident causes. Since the role of 
human factors is often a gray area, it can seldom be accommodated by the rules of 
evidence favored by investigating authorities. Emphasis is placed upon direct cause- 
effect relationships, and causal statements become official designators of blame. The 
inclusion of contributing factors to causal statements perpetuates a distinction between 
primary causes and contributing factors that has the effect of lessening their relationship 
to the accident. McAdams, a senior and respected member of the National Transportation 
and Safety Board (NTSB) commented after a 1978 mid-air collision investigation that: "A 
contributing factor is not a primary cause; it is more remote and does not carry the same 
weight or implications as that of a probable cause." (Bruggink, 1996) As a result, the 
context of the situation is dismissed easier than the individual acts themselves. 
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Zotov (1996) also concludes that there exists reluctance to closely analysis human 
factors related causal factors. Often investigations lead to personal attacks, legal 
obstruction and corporate pressure on the investigating authority if the entire "system" is 
under investigation. A corporation will attempt a legal response that frees them from 
culpability by using the "chain of causation" approach to accident occurrences. 
Additionally, the legal concept of "remoteness of damage" can make it difficult to present 
the concept of a complex network of interacting events that caused an accident. 

Mayer & Ellingstad (1992) state that accident databases frequently describe 
attributes of the physical environment and equipment, but that detailed analysis of 
accident causes, including human factors information, are frequently not represented 
because they are too difficult to obtain and code. Engineers and front-line operators 
design most accident reporting systems with limited backgrounds in human factors. This 
results in a lack of suitability for addressing human factors issues. Naval Aviation is 
continuing to attempt addressing this shortcoming through the Naval Aviation Safety 
Program. 

The purpose of the Naval Aviation safety program is to preserve both human and 
material resources in order to enhance operational readiness. In order to accomplish this 
goal, damage and injury must be addressed to mitigate the hazards inherent in flight 
operations (OPNAVINST 3750.6Q, 1997). The use of HFACS in mishap reporting is 
designed to improve the quality of mishap reports, which will in turn improve the data 
available for analysis. With a higher quality database available, mishap analysts can 
isolate recurring causal factors and recommend suitable strategies to eliminate or reduce 
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the frequency of these accidents. These intervention strategies can then be evaluated for 
their effectiveness in improving operational readiness and reducing budgetary losses. 

D. MISHAP INTERVENTION STRATEGIES 

Mishap intervention strategies are designed to address common mishap causes 
and decrease the probability of their occurring. Typically, these strategies fall into one of 
three categories: engineering controls, policy and procedures and individual protection 
measures. Engineering controls deal with a physical reconfiguration of the system. It 
may be ergonomic, or mechanical in nature. The result of the change is improved system 
performance. Policy and procedures address the circumstances surrounding operations. 
Within the HFACS model, these strategies reflect mishaps in the unsafe conditions and 
unsafe supervision. Changes in flight prerequisites and improved time to train are 
examples of policy modifications. Individual protection refers to the equipment a person 
may use to decrease the possibility of physical injury due to operations. This study 
addresses seven mishap intervention strategies. 

1. Aeromedical Screening and Education 

Among the recommendations made by Schmidt & Parker (1995) several are 
associated with aeromedical screening and monitoring. Currently, this is taking place 
prior to assignment to UAV training. However once initial training is complete, a flight 
surgeon is not assigned to each UAV unit. There still exists a perception that UAV 
crewmembers are just like any other ground community, where crew rest and other 
requirements are perceived as a luxury and not a requirement. Additionally, ongoing 
training is not routinely conducted to address the affects of nutrition, alcohol and tobacco 
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use, stress and psychological readiness on mission performance. The first intervention 
strategy addressed is continuing and increasing the education and follow up attention to 
aeromedical issues and their effect on readiness. This intervention strategy is a policy and 
procedure change and is modeled by the unsafe condition (UC) causation category. 

2. Aircrew Coordination Training / Crew Resource Management 

The policy and procedural recommendation for mishap intervention is increased 
aircrew coordination training (ACT)/CRM. This is being conducted in UAV units; 
however, the lack of experience among crewmembers limits its effectiveness. Because 
UAV operators are cyclically new to the community, there is no senior leadership that can 
speak with a voice of experience to situations that may occur during a flight. Aviators on 
the crew bring a generic ACT background to a unit, but lack specific UAV applications. 
This results in a reactive rather than deliberate approach to an emergency situation. 
Improved, standardized training will impact mishaps caused by unsafe acts (UA) and 
unsafe conditions (UC). 

3. UA V Flight Simulator 

A UAV mission simulator can further enhance mission performance skills and 
mitigate unsafe actions during the conduct of a flight. Schmidt & Parker (1995) find that 
59% of mishaps occurred as a result of electrical, mechanical or engine failure. If these 
situations can be replicated via simulation, a crew can rehearse procedures to correct the 
problem, or put the aircraft in an attitude where damage is minimized. Currently the 
Pioneer does not have a simulation capability where an instructor can induce such an 
emergency, observe the crew response, and then debrief the crew on their performance. 
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The only time the entire crew can operate as a team to address an emergency is during an 
actual flight. A replacement to the GCS is under development as the Tactical Control 
System (TCS). The TCS is designed to be a universal ground control and 
communications shelter that has the ability to interface with all follow-on DoD UAV 
systems. One of the TCS’s requirements is the ability to conduct simulator training 
without conducting an actual mission. The model addresses unit training procedures and 
is modeled in the unsafe act (UA) category. 

4. Automated Take off and Landing System 

Schmidt & Parker attribute 32% of mishaps to take-off and landing error. 
Although not currently a requirement for future UAV systems, several contractors are 
developing an automated take-off and landing system. Their concept is to fly the UAV to 
a predetermined handover point where it flies into a radio beacon. The beacon then takes 
over sending navigational information to the UAV until it is safely on deck. There would 
also be a manual override system. This engineering control will lower the incidents of 
unsafe acts (UA) during the takeoff and landing phase of the flight. 

5. Personnel Stabilization 

In order to lessen the effects of unsafe supervision as well as unsafe acts. The 
UAV community can establish officer specialty codes and career progression 
possibilities. The current situation of constantly rotating first time supervisors into UAV 
units causes instability and limits the "corporate knowledge" of squadron members. If 
implemented, supervisory and performance skills will dramatically improve. The policy 
change is modeled by the unsafe acts (UA) and unsafe supervision (US) categories. 
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6. Engine Replacement 

Although not human factors related, engine failure has been attributed to one 
quarter of UAV mishaps (Schmidt & Parker, 1995). Engineering modifications replacing 
the current engine with a more reliable engine is modeled in the analysis section. 

7. Electronic Waterproofing 

Like engine failures, electronic failures account for another quarter of UAV 
mishaps. Although not originally designed as a naval UAV, the Pioneer is operated by 
the U.S. naval services and would benefit by environmental shielding. The affect of 
engineering modifications such as water proofing system components for operation at sea, 
in the littorals, and in foul weather is addressed. 

E. STOCHASIC MODELING 

Ross (1997) states that in making a mathematical model for a real world 
phenomenon, it is always necessary to make simplifying assumptions so as to make the 
mathematics tractable. However, making too many simplifying assumptions can make 
our conclusions not applicable to the real world. Therefore, the stochastic model must 
strike a balance between simplicity and realism. 

Law & Kelton (1991) state that mathematic model simulation is one of the most 
widely used techniques in operations research. The mathematical model is used to 
represent a system in terms of logical and quantitative relationships that can be 
manipulated to see how the model reacts. A stochastic process is the collection of 
random variables ordered over time, which are defined on a common sample space. A 
stochastic simulation model takes random variable input components and repeats the 
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simulation multiple times in order to achieve a random, although converging solution. 
The output of a stochastic model is itself random, but the number of repetitions can 
decrease the variance in the results. 

In order to create a stochastic simulation model, probability distributions and 
parameters must be identified. In lieu of using the empirical distribution that may contain 
some "irregularities," particularly if the sample size is small, a theoretical, parametric 
distribution is used to smooth out the data (Law & Kelton, 1997). Mintz (1954a) models 
taxi cab accidents in order to determine whether they could be modeled by a specific, 
known distribution. In order to accomplish this simplification, Mintz makes two 
assumptions: (1) accident liability (or proneness) of people is not changed by accidents in 
which they are involved and do not vary over time; and (2) accident liability is distributed 
in some known manner. Through his study of over 1200 taxi cab accidents, Mintz 
(1954b) concludes that accident rates closely approximate a Poisson Process because (1) 
they do display a "memory less" property; and (2) they are distributed as an exponential 
random variable. For the purpose of the UAV mishap model, parameterized distributions 
are tested to determine their suitability as model inputs. 

The goal of the stochastic model is to probabilistically simulate annual Pioneer 
UAV flight operations for the Navy and Marine Corps to approximate mishap events, 
their cost, and effect on readiness. The model simulates these mishaps as a Poisson 
Process. The model is designed with an open architecture to model any durations of 
flight operations, or quantify other measures of performance (MOPs). While it may not 
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be feasible or desirable to modify the Pioneer UAV with these results, they are applicable 
for addressing the next generation replacement system. 

In order to model intervention strategies, this model isolates causal factor 
categories. Although, none of these intervention measures can be totally successful when 
employed in isolation within a mishap category. For example, ACT relies on experience, 
which is tied to specific UAV occupational fields and career progression within the 
community. Simulators also enhance training and readiness. Aeromedical readiness is 
tied to standardization and supervision. All aspects of mishap prevention are 
interdependent. 

Stochastic modeling is used effectively in previous studies to effectively model 
accidents and their effects on cost and missed working hours. Schmorrow (1998) uses 
the HFACS taxonomy to code aviation maintenance mishaps, and stochastic modeling to 
predict cost and readiness. Sciretta (1999) uses a similar methodology to stochastically 
model U.S. Navy shipboard electrical shock mishaps. Teeters (1999), applying HFACS, 
studies the distribution of major and minor aviation maintenance mishaps for Fleet 
Logistic Support (VR) Wing aircraft. This study applies a similar framework to 
stochastically model UAV mishaps. 

F. SUMMARY 

In order to effectively analyze mishap intervention techniques, effective coding 
and documentation is required. The Naval Aviation Safety program provides the 
framework and resources to collect this data. HFACS is the most recent improvement to 
aid in accident investigation, reporting and analysis. HFACS is based upon accepted 
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accident causation theories. Using HFACS coding, mishaps can be categorized, and 
mishap probabilities determined. To analyze these categories, a stochastic model can be 
used to simulate mishap occurrences over a defined period of time. While random in 
themselves, the results are used to weigh the various options available to reduce mishap 
cost and increase mission readiness. 

Applying the HFACS taxonomy, accidents happen as a result of a confluence of 
weaknesses in all four tiers of the model. Intervention strategies are designed to reinforce 
the cohesion of the tiers and decrease the probability of a window of accident 
opportunity. Intervention strategies are designed to strengthen safety environment at each 
of the four levels. 
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III. METHODOLOGY 



A. RESEARCH DATA 

The goal of this research is to use a stochastic model simulation to predict the 
effects of mishap intervention strategies on both operational readiness and budgetary 
costs. The data inputs must be determined in order to conduct the simulation. First, a 
database is constructed from the mishap reports, and causal factors are coded and parsed 
from the data. Analysis of the data leads to the statistical determination of the model 
inputs: inter-event times for mishaps; the mishap rate parameters; the probability of 
mishaps by class (A, B or C); the mishap cost distribution; and the annual UAV flight 
hours. Once calculated, these statistics are used in a stochastic model simulation of 
annual UAV flight operations. The output of the model is expressed as a mission 
readiness factor, and annual mishap costs. Detailed description of each phase is 
discussed below. 

1. Mishap Records 

The Naval Safety Center (NSC) is responsible for maintaining aviation mishap 
records for the Navy and Marine Corps. The foundation of the mishap record is the 
mishap investigation report (MIR). MIRs are required for all Class A, B and C mishaps 
in accordance with OPNAVINST 3750. 6Q. The following items are described in the 
MIR: the events leading up to a mishap, the location and type of operations involved in 
the mishap, causal factors and recommendations for reducing the risk of similar type 
accidents occurring. Prior to FY93, UAV mishaps were not incorporated into the 
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Aviation Safety Program. As a result, the MIRs are not complete and normally point only 
to one causal factor, usually mechanical, electrical or human error. MIRs submitted since 
October 1992 have improved mishap records significantly. Primary and contributing 
factors are reported with greater detail. 

2. Data Base 

The database for this thesis is constructed from the UAV MIRs maintained by the 
NSC, and formatted into an EXCEL spreadsheet. Each mishap event contains the 
following categories: mishap date; air vehicle number; unit; mishap summary; aircraft 
equipment and damage; repair cost; time, location, altitude, and weather at the mishap 
occurrence; causal factors; and recommendations. All repair costs are converted into 
FY98 dollars using the aviation price inflation indices provided by the Naval Center for 
Cost Analysis (NCCA). 

This study addresses mishap rates from the entire database (FY86-FY98). When 
detailed analysis of causal factors is required, it limits its scope to the FY93-FY98 MIRs 
that are standardized by the Aviation Safety Program. Additionally, this partition helps to 
focus analysis on recent steady state UAV operations. Initial mishaps caused by the 
introduction of the air vehicle into the Fleet inventory or by the aberrations to normal 
operations such as those occurring during Operations Desert Shield and Desert Storm do 
not confound the data. 

3. Causal Factors 

A single mishap typically has several codes associated with it. This analysis goes 
beyond the primary causal factor, and addresses known contributing factors. Also of 
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note, mishap coding is done at the lowest level possible given the data provided. For 
example, a UAI (unsafe act: intended) is a subset of UA (unsafe act). However, if the 
MIR provides limited information, coding is done at the highest level discemable. The 
UAV mishaps are coded by causal factor in accordance with the Human Factors Mishap 
Classification System (HFACS) (Shappell & Wiegmann, 1997). Material failures are 
recorded by causal factor. Table 2 contains the codes used in the database: 



CAUSAL FACTOR 


CODE 


Human Factors 


HF 


Unsafe Act 


UA 


Intended 


UAI 


Mistake 


UAIM 


Violation 


UAIV 


Unintended 


UAU 


Slip 


UAUS 


Lapse 


UAUL 


Unsafe Condition 


UC 


Aeromedical 


UCA 


CRM 


UCC 


Readiness 


UCV 


Unsafe Supervision 


US 


Unforeseen 


usu 


Foreseen 


USF 


Electro - Mechanical 


EM 


Engine 


ENG 


Electrical 


ELEC 


Launcher Failure 


LNCHR 


Net Recovery Failure 


NET 


Software 


SOFT 


Other 


OTHER 


Unknown or unspecified 


UNK 



Table 2: Mishap Database Causal Factor Codes 
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B. DATA ANALYSIS 



1. Inter-event Times 

The computation of mishap inter-event times requires that a transformation be 
made from inter-event days, as recorded by the MIR, to inter-event flight hours, for use 
by the model. In order to accomplish this transformation, annual flight hours are assumed 
to be uniformly distributed throughout the fiscal year. An estimate is made of daily flight 
hours using annual flight hours flown information. This daily flight hour rate is 
multiplied by inter-event days to transform inter-event times from days to flight hours 
flown. 

2. Mean , Variance , and Rate Parameters 

Unbiased estimators for the mean and variance of inter-event times are determined 
for each mishap category that is modeled by the simulation. The mishap rate (X) is 
calculated by taking the inverse of the mean inter-event time (Ross, 1997). The 
Kolmogorov-Smimov goodness of fit (KS g.o.f.) test is used to decide whether an 
exponential distribution with parameter (7.) is an appropriate model for the inter-event 
data, and thus whether the occurrence of mishaps could be modeled as a Poisson Process 
(Conover, 1999). The mishap rate for the entire data set and the mishap rates by causal 
factor are the components of the stochastic counting process, which is being modeled. 

3. Probability of Mishap by Class 

The number of mishaps by class (A, B, C) is recorded for each mishap category. 
The probability of Class A, B and C mishaps is estimated by the number of mishaps in 
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each class divided by the total number of mishaps in that category. A vector of Class A, 
Class B and Class C probabilities is used as a multinomial input to the model simulation. 

4. Cost Distribution 

A cost distribution of each mishap class is determined by estimating the mean and 
variance, and then performing a K-S g.o.f. test to confirm their suitability. The combined 
mishap rates, probability of accident severity and cost distributions are used to describe 
the model in terms of a compound Poisson Process. 

5. Determining Annual Flight Hours 

A regression analysis of annual flight hours by fiscal year is used to predict annual 
flight hours for the stochastic model. The model uses the FY99 flight hour prediction as 
the input. The time is the bound in the simulation for the mishap generation period. 

C. SIMULATING THE EFFECTS OF MISHAP INTERVENTION 

The mishap simulation model inputs are the number of simulation repetitions, the 
mishap rate parameter, the multinomial probability vector of the mishap categories (A, B 
and C), and the time period to be simulated. The program instantiates two vectors to 
store the mission readiness factor, and the budgetary cost value. Each simulation run 
goes from time zero to the end time period input value. Time steps are made as a mishap 
is generated using the exponential distribution with the appropriate input rate parameter. 
Once a mishap is encountered, its class is determined randomly, using the input 
probability vector. Mishap cost is calculated using the predetermined cost distribution 
data for that particular mishap class which is "hardwired" into the code. 
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The simulation run ends when the time clock exceeds the input time period. The 
mission readiness factor is calculated by multiplying the number of class A mishaps by 
three, multiplying the number of class B mishaps by two, and adding in the number of 
class C mishaps. If this readiness factor exceeds 21 for the time period, the mission 
readiness is stored as a zero; otherwise, the simulation returns a one - mission ready. 
Both the cost and mission readiness factor are stored in their respective vectors. Once the 
simulation has completed the required number of runs, the simulation returns the mean 
and standard deviation of budgetary cost (FY98$M), and the average of the readiness 
factor. The mishap intervention model code, MishapSim() is programmed in S-Plus 4.0 
and is attached as Appendix B. 

For each mishap intervention strategy, the model is run one thousand repetitions 
and through a total of four simulations. The first simulation is baseline simulation using 
the current mishap rate. A reduction of the mishap rate by 10%, 30% and 50% for each 
intervention strategy is hypothesized for the next three simulation runs. If two or more 
mishap categories are being modeled together, for example, unsafe acts and unsafe 
supervision, the rate parameters are determined as described above. However, all 
replicated mishaps are removed from the composite mishap category. These calculations 
will enable fleet users and program managers to weigh the effectiveness of a proposed 
change with the resulting cost and readiness savings. 
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IV. RESULTS 



A. OVERVIEW 

This chapter presents a database summary for annual flight operations by mishap 
class, and flight hours flown. Mishap causal frequencies are summarized for operations 
going back to 1986, and for the period of the study, 1993-98. Mishap coding results are 
presented by both number and frequency. All simulation model input parameters are then 
estimated and presented. Simulation results for the each mishap intervention strategy are 
presented for the existing baseline and for mishap frequency reductions of 10 percent, 30 
percent and 50 percent. The chapter concludes with a graphic comparison of cost and 
readiness results for all intervention categories. 

B. BACKGROUND UAV FLIGHT DATA 

Figure 7 is a graph of flight hours flown and the number of mishaps versus time. 
Table 3 contains a summary of the UAV annual flight hours, mishaps and associated rates 
for the period since the fielding of the Pioneer system by the Navy and Marine Corps in 
1986. In general, the annual flight hours flown is increasing and the mishap rate is 
decreasing. During Operation Desert Shield and Desert Storm, six UAV units flew just 
under 1000 flight hours. These units deployed with thirty air vehicles. At current mishap 
rates and during a deployment of equal duration, approximately one third of the air 
vehicles would be destroyed or damaged by flight mishaps. This prediction demonstrates 
unacceptable mission readiness and strains maintenance and repair capabilities. 
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Figure 7: Graph of Mishap Rate and Annual flight hours 



Year 


Flight 

Hours 


Class A 


Class B 


Class C 


Total 


Mishap 

Rate 

(per 1000 hours) 


1986 


96.3 


2 


0 


3 


5 


51.92 


1987 


447.1 


7 


0 


2 


9 


20.13 


1988 


1,050.9 


5 


0 


20 


25 


23.79 


1989 


1,310.5 


9 


0 


12 


21 


16.02 


1990 


1,407.9 


5 


1 


15 


21 


14.92 


1991 


2,156.6 


12 


7 


10 


29 


13.45 


1992 


1,179.3 


3 


9 


7 


19 


16.11 


1993 


1,275.6 


1 


5 


3 


9 


7.06 


1994 


1,568.0 


5 


5 


6 


16 


10.20 


1995 


1,391.3 


1 


4 


11 


16 


11.50 


1996 


1,500.5 


9 


9 


5 


23 


15.33 


1997 


2,077.0 


3 


2 


10 


15 


7.22 


1998 


1,972.3 


5 


6 


4 


15 


7.61 


Total 


17,433.3 


67 


48 


108 


223 


12.79 


Class Rate 




3.84 


2.75 


6.20 


12.79 





Table 3: UAV Mishaps by Year and Classification 
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C. MISHAP CODING 

The frequency of mishap cause occurrences is identified in Table 4. This data is 
consistent with the previous studies of Schmidt & Parker (1995) and Seagle (1997). 
Appendix C contains the complete mishap-coding database. Of note, the frequency of 
human factors related mishaps is not increasing. Rather, the reporting of human factors 
related mishaps is increasing in detail. The Mishap Investigative Reports (MIRs) 
submitted since FY 93 have increased the information available upon which to assign 
causal factors. Table 5 is a summary of each category of mishap causation. The 
simulation model evaluates seven mishap intervention strategies. Each intervention 



strategy and its associated mishap classification category are identified in Table 6. 



Mishaps 


Class A 


Class B 


Class C 


Total 


Percentage 


FY 86-98 


Overall 


67 


48 


108 


223 


100% 


Human Factors 


15 


24 


48 


87 


39% 


Electro-Mechanical 


55 


30 


65 


150 


67% 


FY 93-98 


Overall 


24 


31 


38 


93 


100% 


Human Factors 


11 


20 


24 


55 


59% 


Electro-Mechanical 


17 


18 


21 


56 


60% 



Table 4: Mishap Causation Frequency 
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FY 86-98 


FY 93-98 


CAUSAL FACTOR 




CODE 


# 


FREQ 


# 


FREQ 


Human Factors 




HF 


87 


39.0% 


55 


59.1% 


Unsafe Act 




UA 


35 


15.7% 


35 


37.6% 




Intended 


UAI 


16 


7.2% 


16 


17.2% 




Mistake 


UAIM 


11 


4.9% 


11 


11.8% 




Violation 


UAIV 


6 


2.7% 


6 


6.5% 




Unintended 


UAU 


19 


8.5% 


19 


20.4% 




Slip 


UAUS 


13 


5.8% 


13 


14.0% 




Lapse 


UAUL 


3 


1.3% 


3 


3.2% 


Unsafe Condition 




UC 


37 


16.6% 


37 


39.8% 




Aeromedical 


UCA 


9 


4.0% 


9 


9.7% 




CRM 


UCC 


26 


11.7% 


26 


28.0% 




Readiness Violation 


UCV 


9 


4.0% 


9 


9.7% 


Unsafe Supervision 




US 


40 


17.9% 


40 


43.0% 




Unforeseen 


USU 


14 


6.3% 


14 


15.1% 




Foreseen 


USF 


11 


4.9% 


11 


11.8% 


Electro - Mechanical 




EM 


158 


70.9% 


64 


68.8% 




Engine 


ENG 


52 


23.3% 


23 


24.7% 




Electrical 


ELEC 


59 


26.5% 


20 


21.5% 




Launcher failure 


LNCHR 


8 


3.6% 


2 


2.2% 




Net recovery failure 


NET 


16 


7.2% 


7 


7.5% 




Software 


SOFT 


7 


3.1% 


5 


5.4% 




Other 


OTHER 


20 


9.0% 


8 


8.6% 


Unknown / unspecified 




UNK 


8 


3.6% 


7 


7.5% 



Table 5: Mishap Frequency by Causation Code 
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Mishap Intervention Strategy 


Mishap Category 


Aeromedical Screening and Education 


Unsafe Conditions (UC) 


Aircrew Coordination Training / Crew 
Resource Management 


Unsafe Acts, Unsafe Conditions (UA/UC) 


Right Simulator / TCS 


Unsafe Acts (UA) 


Automatic Takeoff and Landing System 


Unsafe Acts (UA) 


Personnel Stabilization 


Unsafe Acts, Unsafe Supervision (UA/US) 


Engine Upgrade 


Engine (ENG) 


Weatherizing the Air vehicle 


Electronic (ELEC) 



Table 6: Mishap intervention Strategy and Associated Causal Category 



D. STOCHASTIC MISHAP MODEL ESTIMATES 

1. Inter-event Times 

Figures 8 through 13 are histogram plots of the inter- arrival times for each 
partition of mishap causal factors. Overlaid on the chart is a rescaled probability density 
function (pdf) of the hypothesized exponential distribution. Table 7 summarizes the 
mean, standard deviation and rate parameters for the mishap category parameter 
estimates. Additionally, 95%, two-sided confidence intervals are presented for each rate 
parameter estimate. Note that the two confidence limits are not equidistant from the point 
estimate. This is due to the lack of symmetry in the exponential distribution. The 
Kolomogorov-Smimov goodness of fit (KS g.o.f.) test for the exponential distribution is 
based on the estimated rate parameter. With a significance level (a) set at 0.05, the KS 
g.o.f. test fails to reject that any of the distributions are exponential. 
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Figure 8: Histogram Plot of UC Data 




Figure 9: Histogram PlotofUA/UC Data 
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Figure 10: Histogram Plot of UA Data 




Figure 1 1 : Histogram Plot of UA/US Data 
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Figure 12: Histogram Plot of ENG Data 
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Figure 13: Histogram Plot of ELEC Data 
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Estimated 

Mean 


Estimated 
Std Dev 


Estimated 

Rate 


Lower 

Cl 


Upper 

Cl 


KS g.o.f 
test 


Category 


n 


flight hours 


flight hours 


Mishaps / 
1000 fit hrs 


mishaps / 
1000 fit hrs 


mishaps / 
1000 fit hrs 


p-value 


UC 


37 


238.6 


216.2 


4.19 


2.95 


5.65 


0.665 


UA/UC 


47 


195.3 


189.8 


5.12 


3.76 


6.68 


0.406 


UA 


35 


262.3 


212.7 


3.81 


2.66 


5.18 


0.160 


UA/US 


52 


176.5 


156.9 


5.67 


4.23 


7.31 


0.484 


ENG 


23 


406.9 


479.5 


2.46 


1.56 


3.56 


0.478 


ELEC 


20 


467.6 


453.8 


2.14 


1.31 


3.17 


0.409 



Table 7: Parameter Estimates for Mean, Standard Deviation and Rate 



2. Mishap Class Probability Parameters 

Table 8 is a summary of the calculations used to estimate the probability of each 
type of mishap (p A , Pb, pc)- Of note, the estimated probability of each class of mishap 
(pA-Hat, pB-Hat, Pc-Hat) varies with the mishap category. For example, an engine failure is 
most likely to cause a catastrophic class A mishap, while the results of an unsafe act can 
be mitigated by other actions of the crew. 







Class A 


Class B 


Class C 


Code 


n 


# mishaps 


PA-Hat 


# mishaps 


PB-Hat 


# mishaps 


PC-Hat 


UC 


37 


6 


0.162 


16 


0.432 


15 


0.405 


UA/UC 


47 


8 


0.170 


19 


0.404 


20 


0.426 


UA 


35 


5 


0.143 


15 


0.429 


15 


0.429 


UA/US 


52 


10 


0.192 


20 


0.385 


22 


0.423 


ENG 


23 


9 


0.391 


4 


0.174 


10 


0.435 


ELEC 


20 


4 


0.200 


10 


0.500 


6 


0.300 



Table 8: Mishap Class Probabilities 
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3 . 



Cost Parameters 



Table 9 summarizes the parametric estimates for the mishap cost distributions. 
Each is hypothesized to be normally distributed. All cost figures are calculated as FY98 
dollars. Figures 14 through 16 are normal probability plots of the actual data versus the 
hypothesized distributions. Graphically, there are some discrepancies between the sample 
data and the hypothesized distribution. However, the KS g.o.f. tests the hypothesized 
distribution using the parameter estimates listed in Table 9. With a significance level (a) 
set at 0.05, the KS g.o.f. test fails to reject the normality of any of the cost distributions. 





Estimates 


Distribution 


KS g.o.f. 


Mean 


Std Dev 


$ FY98 


p-value 


Class A 


$811,504 


$189,306 


N(812K, 187K) 


0.4802 


Class B 


$479,933 


$214,503 


N(480K, 214K) 


0.6073 


Class C 


$87,649 


$64,065 


N(88K, 64K) 


0.1928 



Table 9: FY86-FY98 Mishap Class Cost Distribution 
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Normal Quantiles 
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Figure 14: Normal Probability Plot of Class A Mishap Costs 




Figure 15: Normal Probability Plot of Class B Mishap Costs 
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Figure 16: Normal Probability Plot of Class C Mishap Costs 



4. Annual Flight Hours 

A linear regression of annual flight hours versus the natural log of fiscal year is 
performed on historic flight hours flown. The equation-projected estimate of flight hours 
for the next fiscal year ( FY99 ) is 1,930 flight hours. This will be used as the number of 
annual flight hours in the simulation. Figure 17 contains a graph of the historic flight 
hours and the fitted equation, adjusted to time on a linear scale. 
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Figure 17: Results of Annual Flight Hour Regression 

E. STOCHASTIC MODEL SIMULATION 
1. Model: Baseline of total mishaps 

The aggregate mishap model is presented in Table 10. It is used as a baseline for 
comparison of the remaining models. Calculating the defined readiness factor, this model 
indicates that UAVs never achieve a mission ready condition. Also, UAV mishap costs 
typically exceed $10 million. The following model simulations are used to gain insight 
into the cost and mission readiness improvements over current baseline conditions made 
by targeting mishap causes with the specified strategies. 
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Category: TOTAL 


Baseline 


X - 10% 


X - 30% 


X - 50% 


Mishap Rate / 1000 flight hours 


12.79 


11.51 


8.95 


6.4 


Mean Cost (CY98 $M) 


$10.91 


$9.70 


$7.54 


$5.63 


SD Cost (CY98 $M) 


$2.83 


$2.62 


$2.30 


$1.92 


% change in Cost 




-11.2% 


-30.9% 


-48.4% 


Readiness Index 


0.0000 


0.0020 


0.0640 


0.3000 


% change in Readiness 




N/A 


N/A 


N/A 



Table 10: Aggregate Mishap Model 



2. Model: Increased Aeromedical Screening and Education 
Table 1 1 summarizes the unsafe conditions mishap intervention model. Even at 
current levels, the readiness indicator for unsafe conditions does not go below 80%. This 
is most likely the result of the relative probability of a class A mishaps being low 
compared to the aggregate model. As a result, costs are also kept low, accounting for 
approximately 1/3 of the aggregate mishap costs. 



Category: UC 


Baseline 


X - 10% 


X - 30% 


X - 50% 


Mishap Rate / 1000 flight hours 


4.20 


3.78 


2.95 


2.10 


Mean Cost (CY98 $M) 


$3.51 


$3.09 


$2.60 


$1.94 


SD Cost (CY98 $M) 


$1.44 


$1.36 


$1.21 


$1.03 


% change in Cost 




-12.1% 


-26.1% 


-44.8% 


Readiness Index 


0.8400 


0.9170 


0.9710 


0.9970 


% change in Readiness 




9.2% 


15.6% 


18.7% 



Table 1 1 : Unsafe Conditions Model 
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3. Model: Aircrew Coordination Training / Crew Resource Management 
Table 12 summarizes the mishap intervention model for the aggregate of unsafe 
acts and unsafe conditions. ACT / CRM will have to reduce the mishap rate by over 10% 
in order to get readiness above the 80% level. Additionally, at $4 million per year in 
mishap cost contributions, unsafe acts and unsafe conditions combined contribute to 
approximately one-third of mishap costs. 



Category: UA/UC 


Baseline 


X - 10% 


X - 30% 


X - 50% 


Mishap Rate / 1000 flight hours 


5.11 


4.60 


3.58 


2.56 


Mean Cost (CY98 $M) 


$4.01 


$3.63 


$2.93 


$2.16 


SD Cost (CY98 $M) 


$1.56 


$1.40 


$1.29 


$1.10 


% change in Cost 




-9.5% 


-26.8% 


-46.0% 


Readiness Index 


0.7040 


0.8010 


0.9350 


0.9910 


% change in Readiness 




13.8% 


32.8% 


40.8% 



Table 12: Aggregate Unsafe Acts / Unsafe Conditions Model 



4 . Model: UAV Flight Simulator / Automated Take off and Landing 

System 

Table 13 summarizes the mishap intervention model for unsafe acts. This model 
is used to evaluate the potential results of both the flight simulator and of the take off and 
landing aids. While the category of unsafe acts contributes to 37% percent of mishaps, 
their effect on readiness is not as profound. Even in the baseline case, unsafe acts have a 
ready index of nearly 90 percent. Because of their nature, unsafe acts will contribute to 
mishap occurrences, but their individual effects do not have as great an impact on overall 
changes in readiness and costs. 
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Category: UA 


Baseline 


X - 10% 


X - 30% 


X - 50% 


Mishap Rate / 1000 flight hours 


3.82 


3.44 


2.67 


1.91 


Mean Cost (CY98 $M) 


$3.16 


$2.94 


$2.29 


$1.74 


SD Cost (CY98 $M) 


$1.40 


$1.29 


$1.11 


$0.95 


% change in Cost 




-6.9% 


-27.6% 


-44.8% 


Readiness Index 


0.8970 


0.9430 


0.9860 


1.0000 


% change in Readiness 




5.1% 


9.9% 


11.5% 



Table 13: Unsafe Acts Model 



5. Model: Personnel Stabilization 

Table 14 summarizes the mishap intervention model for the aggregate of unsafe 
acts and unsafe supervision. This is the only model that incorporates the effects of unsafe 
supervision. Although difficult to isolate, unsafe supervision when coupled with unsafe 
acts does have a profound effect on both cost and readiness. The model bears out that 
personnel stability has the potential to significantly improved readiness and reduced cost. 



Category: UA/US 


Baseline 


X - 10% 


X - 30% 


X - 50% 


Mishap Rate / 1000 flight hours 


5.66 


5.09 


3.96 


2.83 


Mean Cost (CY98 $M) 


$4.52 


$4.20 


$3.39 


$2.52 


SD Cost (CY98 $M) 


$1.76 


$1.60 


$1.44 


$1.21 


% change in Cost 




-7.2% 


-25.2% 


-44.4% 


Readiness Index 


0.5570 


0.6760 


0-8710 


0.9750 


% change in Readiness 




21.4% 


56.4% 


75.0% 



Table 14: Aggregate Act/Unsafe Supervision Model 
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6. Model: Engine Replacement 

Table 15 summarizes the mishap intervention model for engine failure related 
events. Engine failure accounts for nearly 25 percent of mishaps. Intuitively, 
modifications to the engine should cause a significant cost reduction and readiness 
improvement. However, the model results indicate that the effect on mission readiness is 
negligible since the baseline readiness factor already exceeds 95 percent. Additionally, 
the budgetary cost of these mishaps is less than or equal to other mishap categories. 



Category: ENG 


Baseline 


X - 10% 


X - 30% 


X - 50% 


Mishap Rate / 1000 flight hours 


2.46 


2.21 


1.72 


1.23 


Mean Cost (CV98 $M) 


$2.64 


$2.44 


$1.90 


$1.49 


SD Cost (CY98 $M) 


$1.29 


$1.28 


$1.15 


$0.92 


% change in Cost 




-7.6% 


-28.0% 


-43.6% 


Readiness Index 


0.9750 


0.9780 


0.9990 


1.0000 


% change in Readiness 




0.3% 


2.5% 


2.6% 



Table 15: Engine Model 



7. Model: Electronic Waterproofing 

Table 16 summarizes the mishap intervention model for the improvements to 
electronic component reliability. Electronic component failure is cited in over 20% of 
UAV mishaps, but similar to engine failure, its impact on readiness and annual costs is 
overshadowed by the other causal categories. The model indicates that the effect of 
electrical failures upon mission readiness is minimal. 
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Category: ELEC 


Baseline 


X. - 10% 


X-30% 


X - 50% 


Mishap Rate / 1000 flight hours 


2.14 


1.93 


1.50 


1.07 


Mean Cost (CY98 $M) 


$2.20 


$2.06 


$1.70 


$1.34 


SD Cost (CY98 $M) 


$1.07 


$1.01 


$0.91 


$0.83 


% change in Cost 




-6.6% 


-23.0% 


-39.2% 


Readiness Index 


0.9980 


0.9980 


1.0000 


1.0000 


% change in Readiness 






0.0% 


0.2% 


0.2% 



Tabue 16: Electronic Model 



F. MODEL COMPARISON 

Figures 18 and 19 are a comparison between the results of the mishap intervention 
simulations. Figure 1 8 compares the cost reduction in each category for the four runs of 
the model. Clearly, there is improvement across all strategies, but improvements in 
UA/UC and UA/US appear to cause more significant cost savings. Figure 19 illustrates 
the mission readiness index improvement. Inspection of the graph reveals that addressing 
the UC, UA/UC and UA/US categories will have the greatest impact on mission 
readiness. According to the simulation, none of other causal factors reduces the readiness 
factor below 80 percent. 
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Readiness Index FY98 $M 



□ Baseline MX- 10% □ X - 30% □ X - 50% | 




Mishap Category 



Figure 18: Mishap Cost Reduction 



□ Baseline mX-\0% DX-30% CU-50% 




TOTAL UC UA/UC UA UA/US ENG ELEC 

Mishap Category 



Figure 19: Mishap Readiness Index Improvement 
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V. CONCLUSIONS AND RECOMMENDATIONS 



A. MISHAP CLASSIFICATION 

Unmanned Aerial Vehicle (UAV) mishaps are successfully categorized by the 
Human Factors Accident and Classification System (HFACS) using the Naval Aviation 
Safety Program (OPNAVINST 3750.6Q) mishap report procedures. However, 
granularity at the lowest tiers of the HFACS model can be difficult to ascertain. The 
information available for analysis appears to vary with the writer of the report. If that 
individual is aware and comfortable with HFACS, the report tends to address all of the 
details required for future analysis. Otherwise, reports contain generalities and lack the 
specific details necessary to conduct an in depth analysis of the data. The planned 
incorporation of the HFACS taxonomy into the next revision of OPNAVINST 3750.6R, 
should standardize reporting procedures, and educate those preparing mishap reports on 
the scope that human factors can have on flight operations. 

This analysis of mishap classification is conducted independently of the data 
partition of Schmidt & Parker (1995) and Seagle (1997). The three separate mishap 
categorizations are similar in terms of the relative frequency and causes of mishaps. 
Individual judgments were required by those conducting the study in order to place a 
particular mishap into a causation category. While not always agreeing in the exact 
classification, the three studies do conclude that human error is at least partially 
attributable for approximately one half of UAV mishaps. Engine and electronic failure 
each account for 20 to 25 percent of mishaps. Finally, although the mishap rate is 
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decreasing, current mishap damage and losses will continue to have a profound affect on 
mission readiness. 

B. MISHAP MODELING 

The mishap occurrence distribution is modeled effectively as a Poisson process. 
Cost data and mishap class data distributions can also be modeled by specific 
distributions. These distributional quantities are effectively input into the stochastic 
model, allowing the analyst to simulate annual UAV flight operations accurately. 

C. MODEL RESULTS 

The aggregate UAV mishap rate must be reduced by one half in order to achieve a 
significant change in total mishap occurrences. This mishap reduction still only raises the 
readiness index to 0.300. (See Figure 19.) In order to focus the mishap intervention 
strategies, they must be analyzed in isolation. Under the current baseline rates two 
mishap categories, Unsafe Acts and Unsafe Conditions (UA/UC), and Unsafe Acts and 
Unsafe Supervision (UA/US), cause the readiness index to fall below 0.800. The 
simulations demonstrate that improvements up to 40.8 percent and 75.0 percent, 
respectively, can be achieved in these categories. This indicates that they should be 
considered primary targets for intervention strategies. 

The other impact of the simulated UAV mishaps is cost. Again, the UA/UC and 
UA/US categories are the two most costly mishap causal factors. Each contributes 
greater than $4 million (FY98$) to annual mishap costs in the model. Intervention in 
these categories can reduce costs by 44.4 percent and 40.8 percent, respectively. 
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Although not considered in this model, intervention strategies should be compared to find 
the best value for the money invested. 

Contrary to expectations, incorporating engineering modifications, (engine 
improvement/replacement, and electronic waterproofing) will have marginal effects on 
readiness and cost. At current mishap rates, the engine and electronic configuration do 
not degrade mission readiness below 95 percent. Additionally, their mishap costs are 
approximately the same as the other mishap categories. The cost involved in research, 
development and procurement will most likely exceed current mishap cost predictions 
and can be better spent improving other aspects of the Pioneer system. 

D. RECOMMENDATIONS 

The evaluation of the stochastic model results points to mishap intervention 
measures for the UA/UC and UA/US categories. This research proposes improvements 
to aircrew coordination training and crew resource management in order to alleviate the 
effects of the UA/UC category. Initiatives in this area will not only improve Pioneer 
UAV operations, but will have a listing impact on the UAV community, regardless of the 
system employed. When the Pioneer is eventually replaced, only minor modifications 
should be necessary to adjust to the idiosyncrasies of the new systems. The second area 
of intervention recommended is UAV personnel stabilization. The UAV community is 
still relatively new to Naval Aviation operations. As such, the community needs to 
mature. Unit leaders, both officer and enlisted, should have experience and knowledge of 
the system in order to effectively manage unit operations and individual crewmembers. A 
UAV career path should be created to track these individuals and assign them 
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appropriately throughout their careers so that the UAV community can benefit from the 
stability and "corporate knowledge" enjoyed by other Naval flight communities. 
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APPENDIX A: PIONEER UAV SYSTEM DESCRIPTION 



A. SYSTEM COMPONENTS 

A Pioneer UAV system consists of five Pioneer air vehicles, a ground control 
station (GCS). a portable control station (PCS), a tracking communications unit (TCU), a 
data link, two remote receiver stations (RRS) and a reconnaissance payload. The system 
can be operated aboard specially configured USS Austin Class Landing Platform Dock 
(LPD-4) ships or from prepared airstrips ashore. 

L The Air Vehicle 




Figure 20: Pioneer air Vehicle 



The Pioneer vehicle air is 14 feet long and is pusher-propeller driven, powered by 
a 26 hp, two stroke, twin cylinder, rear mounted engine, similar to a snowmobile engine. 
The air vehicle is made of fiberglass, Kevlar and other low cost composite materials, and 
weighs 463 lbs. The air vehicle can operate up to an altitude of 15,000 feet, but normally 
flies between 3000 and 5000 feet in order to optimize payload performance. Because the 
air vehicle uses a laminated wood propeller, and the electronic components are not 
weatherized, the UAV cannot fly through visible moisture (fog, clouds, rain, etc.) or icing 
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conditions. The air vehicle has up to four-hour time of flight at a cruise speed of 65 
knots, which normally translates into 2-2.5 hours in an objective area depending on the 
proximity of the airstrip to the objective. 

The air vehicle is launched using one of three methods. The rocket-assisted take 
off (RATO) is the only method available for shipboard operations. A rocket is placed 
under the vehicle to propel it into the air. Having reached sufficient altitude and airspeed, 
the rocket motor shuts down and is jettisoned from the UAV. Land-based units can also 
conduct RATO launch. Additionally on land, the UAV can use a standard rolling take of 
from a 1500-foot runway. Because of restrictive crosswind parameters, or air density 
constraints, a rolling takeoff may not be possible. For these instances, a pneumatic 
launcher mounted on a 5-ton truck can propel the vehicle to the minimum altitude and 
airspeed to transition to vehicle-powered flight. 




Figure 2 1 : RATO Takeoff 
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FIGURE 22: PNEUMATIC LAUNCHER TAKEOFF 



There are two ways to safely recover the air vehicle. Operations at sea require the 
UAV to be flown into a large net suspended across the aft part of a ship’s helicopter flight 
deck. Once, in the net, the recovery system collapses around the UAV allowing it to be 
lifted out. Because this recovery method is tantamount to a controlled crash, there is 
frequent damage to the UAV. The second recovery method, used ashore, is an arrested 
recovery by a miniature tailhook on an airstrip. While much more suitable for a crash 
free recovery, cross wind limitation must be monitored in order to assure a successful 
recovery (MAWTS-1, 1997). 



figure 23: UAV Shipboard Landing 
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2 . 



The Ground Control Station (GCS) 



The GCS is the focus of activity for UAV missions. The system can either be 
land based or installed aboard ship. The GCS consists of three electronics bays manned 
by two operators. The pilot bay includes all controls, instruments and displays required to 
safely "fly" the vehicle. The observer bay provides control and display of the imaging 
payloads carried by the UAV. The tracking bay displays the UAV position based on data 
from the TCU and global positioning satellites (GPS) (Pioneer UAV, INC., 1999). 




Figure 24: Inside the GCS 

3. The Portable Control Station (PCS) 

The PCS provides the capability to control the UAV during pre-flight, launch and 
recovery operations, allowing the GCS to locate where it can most effectively conduct the 
mission. Because the air vehicle relies on line of site communications between the 
control station and the air vehicle, split sight operations are common in rugged. 
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compartmented terrain. The PCS provides the ability to control the launch sequence from 
a local airstrip, and then steer the air vehicle to a predetermined handover point. There, 
the GCS, operating from a more advantageous location, can take control of the UAV and 
conduct the mission further down range (MAWTS-1, 1997). 




Figure 25: Inside the PCS Control Bay 

4. The Tracking Control Unit (TCU) 

The TCU shelter contains the UAV communication equipment and antennas. The 
TCU contains a sophisticated, jam resistant, C-band, 100 nmi. range data link. Both the 
video and telemetry link use directional antennas between the air vehicle and the TCU in 
order to ensure video quality and minimize the probability of data link intercept by the 
enemy. The system also has an omni-directional, UHF backup link for redundancy. The 
TCU can be remoted 1000 meters from the GCS by fiber-optic cable, enhancing the 
system's and personnel battlefield survivability (Pioneer UAV, INC., 1999). 
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FIGURE 26: THE TCU 



5. Remote Receiving Station (RRS) 

The ruggedized RRS provides real-time reception of the UAV video picture at 
remote locations. The Marine Corps has mounted the RRS on a high mobility multi- 
wheeled vehicle (HMMW V), a light armored vehicle (LAV) and aboard a UH- 1 N Huey 
helicopter, allowing the tactical commander to have real-time imagery regardless of 
where the command post is locating (MAWTS-1, 1997). 




Figure 27: The RRS 
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6. Reconnaissance Payloads 

The air vehicle can carry one of two separate, gyrostabilized payloads: the MKD- 
200 electro-optical day camera, and the MKD-400(C) forward-looking infrared (FLIR) 
night camera. The MKD-200 E-0 camera can detect targets up to 18 km range, and 
recognizes targets at 3 km range. The MKD-400(C) FLIR camera can detect a target at 8 
km range, and recognizes a target at up to 4 km. Camera performance is enhanced by 
increased thermal differential between the target and the surrounding background 
(MAWTS-1, 1997). 




Figure 28: UAV Payloads 



B. CREW COMPOSITION 

The term "unmanned" is actually a misnomer when applied to the Unmanned 
Aerial Vehicle system because UAV operations involve many participants. The essential 
members of a UAV crew include a Mission Commander, an Internal Pilot, a Payload 
Operator and an External Pilot. Additionally launch and recovery teams and maintenance 



79 



personnel will be involved in flight operations. The responsibilities of each crewmember 
are summarized below (JUAVTOPS, 1997): 

1. Mission Commander (MC) 

The MC is typically a rated Naval Aviator or Naval Flight Officer, who has the 
supervisory responsibility for the UAV mission. This includes organizing the entire flight 
crew, coordination with external agencies and supported units during pre-mission 
planning, execution, and post mission debriefing. 

2. Internal Pilot (IP) 

Typically a senior enlisted aviation rating, who flies the UAV down range, 
monitors instruments to ensure proper operation, and assists the payload operator (PO) to 
get optimal camera position. The IP is also responsible for in-flight emergencies. 

3. Payload Operator (PO) 

The PO is an enlisted operator who controls the UAV camera and monitors the 
tracker bay to insure proper orientation. The PO assists the IP through visual navigation 
and during in-flight emergencies. 

4. External Pilot (EP) 

The EP, typically an enlisted operator, flies the UAV during launch and recovery 
operations. He coordinates the UAV handoff to the IP, and handles all launch and 
recovery emergencies. 

5. Other Crewmembers 

Depending on the complexity of the mission, and the experience of the crew, 
additional personnel may be required to augment the basic crew. Intelligence personnel 
may be involved to exploit the video imagery and pass that information on to the 
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appropriate supported units. If the mission calls for fire support adjustment, an artillery 
or naval gunfire forward observer will be added to the crew. The crew is then rounded 
out with UAV maintenance and communications personnel. 
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APPENDIX B: MISHAP SIMULATION CODE 



MishapSim( ) 

function (runs , lamda, PHat, fltHours) 

{ 

##### 

# Input : 

# runs is the number of repetitions for the simulation 

# lamda is the inter-arrival mishap rate for an exponential 

# distribution. 

# PHat is a vector of probabilities for a class A, B or C 

# mishap as calculated by the ClassSim( ) function. 

# fltHours is the length of the flight period to be 

# simulated. 

# 

# Function: 

# This simulation is based on an annual flight hour period. 

# A random exponential variable with the inputted rate 

# parameter is used to simulate the occurrence of a mishap. 

# This is used to increment the time . 

# 

# Once a mishap is generated, a second random sample is 

# drawn to determine the type of mishap (class A, B or C) . 

# The damage cost is also determined by the cost 

# distribution data that is hardwired into the program. 

# 

# A cost vector and readiness index vector are built as 

# each annual flight period is completed. The weighted 

# readiness index weighs a class A = 3, class B = 2, class 

# C =1. The maximum index in order to be "MISSION READY" 

# is 21. A readiness index greater than 21 indicates "NOT 

# MISSION READY" . 

# 
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# Returns: 

# The Function returns a vector of simulated annual costs, 

# the mean and standard deviation of the mishap cost and 

# the average readiness index for the year. 

##### 

readiness <- vector (" integer " , runs) 
totalCost <- vector ( "double" , runs) 
for(i in 1 : runs ) { 

time <- 0 

mishapClass <- c ( "A" , "B", "C") 

cost <- 0 

numA <- 0 

numB <- 0 

numC < - 0 

while (time < fltHours) { 

time <- time + rexp(l, lamda/1000) 
mishapType <- sample (mishapClass , 1, replace 
= T, PHat) 

if (mishapType == "A") { 

damage <- rnorm(l, 812000, 187000) 
numA <- numA + 1 

} 

if (mishapType == "B" ) { 

damage <- max(200000, rnorm(l, 480000, 
214000) ) 

numB <- numB + 1 

} 

if (mishapType == "C") { 

damage <- max (10000, rnorm(l, 88000, 
64000) ) 

numC < - numC + 1 

} 

cost <- cost + damage 

} 

totalCost[i] <- cost 
84 



} 



readiness [i] <- 1 

readyFactor <- 3 * numA + 2 * numB + numC 
if ( readyFactor > 21) { 

readiness [i] <- 0 

} 



} 

mishapCost <- mean ( totalCost ) 

SDCost <- sqrt (var ( totalCost ) ) 
readylndex <- mean ( readiness ) 

return ( totalCost , mishapCost, SDCost, readylndex) 
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APPENDIX C: MISHAP CODING DATABASE 
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